{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "run_colab_on_devboard.ipynb",
      "provenance": [],
      "collapsed_sections": [
        "zBubcXOfW3Au"
      ],
      "toc_visible": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zBubcXOfW3Au",
        "colab_type": "text"
      },
      "source": [
        "##### *Copyright 2019 Google LLC*\n",
        "*Licensed under the Apache License, Version 2.0 (the \"License\")*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rKwqeqWBXANA",
        "colab_type": "code",
        "cellView": "both",
        "colab": {}
      },
      "source": [
        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dGib_ljiVqZp",
        "colab_type": "text"
      },
      "source": [
        "# Run Colab on a Coral Dev Board\n",
        "\n",
        "This tutorial shows you how to run Jupyter notebooks on your Coral Dev Board, and then edit and execute them from your connected computer using the [Google Colab interface](https://research.google.com/colaboratory/faq.html).\n",
        "\n",
        "By the end of this tutorial, you'll connect this notebook to your Dev Board and run an inference on the board using code from this page.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "viewin-badges",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/google-coral/tutorials/blob/master/run_colab_on_devboard.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open in Colab\"></a>\n",
        "&nbsp;&nbsp;&nbsp;&nbsp;\n",
        "<a href=\"https://github.com/google-coral/tutorials/blob/master/run_colab_on_devboard.ipynb\" target=\"_parent\"><img src=\"https://img.shields.io/static/v1?logo=GitHub&label=&color=333333&style=flat&message=View%20on%20GitHub\" alt=\"View in GitHub\"></a>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gewiVBajCvWv",
        "colab_type": "text"
      },
      "source": [
        "## Requirements"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7MdedD-KZNku",
        "colab_type": "text"
      },
      "source": [
        "+ You need a Coral Dev Board and it should be [set up with the latest software](https://coral.ai/docs/dev-board/get-started/).\n",
        "\n",
        "  If you run `cat /etc/mendel_version` on the Dev Board, it should print 4.0 or higher.\n",
        "\n",
        "+ The computer you're using to read this tutorial must have [MDT installed](https://coral.ai/docs/dev-board/mdt/).\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fmb_p_OkCp0J",
        "colab_type": "text"
      },
      "source": [
        "## Install Jupyter"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qR_kI6EpUJ_u",
        "colab_type": "text"
      },
      "source": [
        "First you need to install Jupyter on your Dev Board as follows:\n",
        "\n",
        "1.  Open a shell on your Dev Board:\n",
        "\n",
        "    ```bash\n",
        "    mdt shell\n",
        "    ```\n",
        "\n",
        "2.  Install the Python 3 development tools on the board:\n",
        "\n",
        "    ```bash\n",
        "    sudo apt-get update\n",
        "\n",
        "    sudo apt-get install python3-dev\n",
        "    ```\n",
        "\n",
        "3.  Install Jupyter:\n",
        "\n",
        "    ```bash\n",
        "    pip3 install jupyter\n",
        "    ```\n",
        "\n",
        "4.  Reload the `.profile` file to access Jupyter's location (at `~/.local/bin`):\n",
        "\n",
        "    ```\n",
        "    source $HOME/.profile\n",
        "    ```\n",
        "\n",
        "5.  Verify it works by running `jupyter --version`.\n",
        "\n",
        "    If this prints `command not found`, then your Mendel version might be out of date or the Jupyter installation failed.\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xNUQayMHCJGP",
        "colab_type": "text"
      },
      "source": [
        "## Start Jupyter on the Dev Board"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AU1M5f8rUh_C",
        "colab_type": "text"
      },
      "source": [
        "Before starting Jupyter on the Dev Board, you need to install `jupyter_http_over_ws`, which enables your host computer to communicate with Jupyter via WebSocket:\n",
        "\n",
        "```bash\n",
        "pip3 install jupyter_http_over_ws\n",
        "\n",
        "jupyter serverextension enable --py jupyter_http_over_ws\n",
        "```\n",
        "\n",
        "Once that's done, start Jupyter as follows:\n",
        "\n",
        "```bash\n",
        "jupyter notebook \\\n",
        "  --NotebookApp.allow_origin='https://colab.research.google.com' \\\n",
        "  --port=8888 \\\n",
        "  --NotebookApp.port_retries=0\n",
        "```\n",
        "\n",
        "This outputs several messages, including a URL such as `http://localhost:8888/?token=...`. You'll\n",
        "need to come back and copy this token later.\n",
        "\n",
        "Be sure you leave this terminal open because it's now running the Jupyter notebook."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JnHd8XHWCXCq",
        "colab_type": "text"
      },
      "source": [
        "## Start SSH port forwarding\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "o3DVvLEfUdhI",
        "colab_type": "text"
      },
      "source": [
        "\n",
        "Although Jupyter is now running, it's only accessible from the Dev Board. To access it from the host computer, you need to forward your localhost traffic to the Dev Board with an SSH tunnel. The effect is to make Google Colab think Jupyter notebook is running on your host computer even though it's actually running on the Dev Board.\n",
        "\n",
        "Open a new terminal on your host computer (where you run `mdt shell`) and run this command:\n",
        "\n",
        "```bash\n",
        "ssh -N -L 8888:localhost:8888 mendel@192.168.100.2 -i ~/.config/mdt/keys/mdt.key\n",
        "```\n",
        "\n",
        "If asked, `Are you sure you want to continue connecting?`, type `yes`.\n",
        "\n",
        "If you see a warning that says `REMOTE HOST IDENTIFICATION HAS CHANGED`, then you need to open the `known_hosts` file and remove the line specifying the key for `192.168.100.2`. Then try again.\n",
        "\n",
        "Now leave this window active because this command is what keeps the SSH tunnel open between the Dev Board and the host computer.\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BfGk4i4WCbPX",
        "colab_type": "text"
      },
      "source": [
        "## Connect this Colab to the local runtime"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zk7ekSr7UYuP",
        "colab_type": "text"
      },
      "source": [
        "Now login to the Jupyter notebook and connect this Colab:\n",
        "\n",
        "1.  Open http://localhost:8888 in a browser on the host computer.\n",
        "    \n",
        "2.  You should see a Jupyter notebook page asking for \"Password or token\" at the top. Paste the token printed by the `jupyter notebook` command above (just the part following `?token=`), and click **Log in**.\n",
        "\n",
        "2.  Now, on *this tutorial page's* Colab interface, click the drop-down arrow next to **Connect** (near the top-right corner of the web page) and click **Connect to local runtime**.\n",
        "\n",
        "4.  Use `8888` as the port and click **Connect**.\n",
        "\n",
        "    The button should now say \"Connected (Local)\".\n",
        "\n",
        "This notebook (the page you're reading) is now connected to your Dev Board and you can run code from this page on the board!\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LmsZVTCBVGtB",
        "colab_type": "text"
      },
      "source": [
        "## Run notebook code on the Dev Board\n",
        "\n",
        "\n",
        "You can execute the code blocks below one at a time or run them all by selecting **Runtime > Run all** in the toolbar.\n",
        "\n",
        "First you need to download pre-compiled models and data from the Edge TPU API examples:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jOKXppy9Km91",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "! sudo apt-get update\n",
        "\n",
        "! sudo apt-get install edgetpu-examples"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pLCue5FEVjPq",
        "colab_type": "text"
      },
      "source": [
        "Then you can run an inference with this code:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "p3A3R8KwL5fj",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from edgetpu.classification.engine import ClassificationEngine\n",
        "from edgetpu.utils import dataset_utils\n",
        "from PIL import Image\n",
        "\n",
        "# Prepare labels.\n",
        "labels = dataset_utils.read_label_file('/usr/share/edgetpu/examples/models/inat_bird_labels.txt')\n",
        "# Initialize engine.\n",
        "engine = ClassificationEngine('/usr/share/edgetpu/examples/models/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite')\n",
        "# Run inference.\n",
        "img = Image.open('/usr/share/edgetpu/examples/images/parrot.jpg')\n",
        "for result in engine.classify_with_image(img, top_k=3):\n",
        " print('---------------------------')\n",
        " print(labels[result[0]])\n",
        " print('Score : ', result[1])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OrkVo6HbuiqO",
        "colab_type": "text"
      },
      "source": [
        "You should see classification results for \"Ara macao (Scarlet Macaw).\" \n",
        "\n",
        "You can now create your own Colab files and connect them to the same local runtime."
      ]
    }
  ]
}
